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COST  GROWTH  IN  NEW  PROCESS  FACILITIES 


Edward  W.  Her row 

This  paper  describes  the  causes  of  cost  misestimation  for  major 
plants  and  speculates  about  why  the  estimation  problem  has  been  so 
resistant  to  resolution.  My  discussion  is  based  on  a  substantial  body 
of  Rand  work  that  developed  methods  for  evaluating  the  cost,  schedule, 
and  performance  of  process  plant  projects . 

Figure  1  displays  both  the  nature  and  the  magnitude  of  the  cost 
misestimation  problem.  This  figure  shows  estimates  for  more  than  AO 
chemical  process  plants  built  from  the  late  1960s  to  the  early  1980s. 
When  we  categorize  the  estimates  according  to  the  project  stage  at  which 
they  were  made,  we  see  the  size  of  the  misestimation  in  early  cost 
estimates,  especially  when  new  technology  is  involved.  Even  after  the 
effects  of  inflation,  unanticipated  regulatory  changes,  and  the  like 
have  been  removed,  estimates  made  before  detailed  engineering  is  well 
advanced  are,  on  average,  very  poor  predictors  of  the  actual  cost  of 
plants.  -Aa  we.  suggest  b«low^>  ^this  problem  of  average  underestimation  is 
but  one  manifestation  of  the  difficulties  of  arriving  at  reasonably 
accurate,  reasonably  early  estimates. 

APPROACH  TO  THE  ANALYSIS 

Our  work  was  prompted  by  the  systematic  estimating  errors  witnessed 
during  the  1970s  for  synthetic  fuels  and  other  energy  process  plants. 

Our  basic  approach  was  to  use  historical  data  to  develop  estimating 
relationships  for  cost,  schedule,  and  performance.  These  estimating 
relationships  statistically  link  system  characteristics  and 
characteristics  of  the  particular  projects  with  what  actually  happened 

lThe  empirical  analyses  underlying  the  discussion  in  this  paper 
were  the  product  of  a  team  of  researchers  af  Rand.  In  particular,  I 
acknowledge  the  role  of  my  coauthors,  K  E.  Phillips  and  C.W.  Myers,  on 
the  report  from  which  this  paper  is  drawn.  See:  F.  .W.  Merrow,  K.E. 
Phillips,  and  C.W.  Myers,  Understand ing  Cost  Growth  and  Performance  . 
Shortfalls  in  Pioneer  Process  Plants,  The  Rand  Corporation,  R-2569-DOE, 
September  1981. 
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COST  ESTIMATION  ACCURACY: 
THE  PROBLEM 
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on  the  projects  in  terms  of  cost  estimation  accuracy,  schedule, 
slippage,  and  plant  performance  after  start-up.  In  this  discussion,  we 
will  consider  only  the  cost  estimation  part  of  the  analysis. 

Developing  a  Model 

When  we  started  the  Pioneer  Plants  Study  five  year  ago,  we  were 
surprised  to  discover  that  there  were  no  publicly  available  empirical 
analyses  of  the  problems  of  cost  growth  in  process  plants.  The. 
literature  was  (and  continues  to  be)  largely  anecdotal.  This  situation 
was  not  unique  to  process  plants:  The  only  systematic  empirical  analyses 
of  cost  growth  that  were  publicly  available  were  studies  of  the 
acquisition  of  major  weapons  systems  by  the  United  States  Government  2 

2See  for  example,  R.L.  Perry  et  al..  System  Acquisition  Strategics, 
The  Rand  Corporation,  R-733-PR/ARPA ,  June  1971;  E.W.  Me r row,  S.W. 

Chapel,  and  C.  Worthing  A  of  Cost  f-'st  imat  ion  in  n.  ;«  Tt-chnn  mg  ms  : 

Implications  for  Energy  Process  Plants,  The  Rand  Corporation, 

R-2AB1-D0F,.  July  1979;  and  Robert  Summers,  Cost  Fst  imatc.s  as  Predictors 
of  Actual  Weapons  Costs:  A  Study  of  Major  Hardware  Articles  The  Rand 
Corporation,  RM-3061-PR,  March  1963. 


These  weapons  acquisition  analyses  were  extremely  important  in 
helping  us  frame  the  right  questions  and  in  guiding  cur  choice  of 
analytic  methods.  While  it  is  too  strong  to  say  that  if  you've  seen  one 
major  technology  development  program,  you've  seen  them  all,  it  is  quite 
clear  that  there  are  fundamental  commonalities  among  high  technology 
development  and  commercialization  programs.  The  problems  faced  by  the 
cost  estimator  in  the  early  stages  are,  we  believe,  generic,  Therefore, 
many  of  the  ideas  and  conclusions  that  have  been  developed  in  one  area 
should  be  transferable  in  broad  outline  to  other  arenas. 

After  about  six  months  of  discussions  with  industry  engineers  to 
help  define  our  data  needs,  we  solicited  data  for  process  plants  using 
the  following  criteria:  (1)  the  plant  had  been  built  after  1965  (we 
believed  that  we  could  not  use  plants  built  earlier  and  still  make 
adequate  adjustments  for  inflation)  and  (2)  the  plant  was  in  the  U.S.  or 
Canada  (again  for  reasons  of  making  constant  dollar  cost  adjustments). 

We  did  express  a  preference  for  p  !  mts  that  had  some  new  technology. 
However,  we  explicitly  asked  that  industry  participants  not  select 
plants  simply  because  they  had  had  problems.  We  ultimately  collected 
data  from  more  than  40  plants  built  by  more  than  three  dozen  companies 
in  the  oil  and  chemical  industry. 

The  composition  of  the  sample  is  shown  in  Figure  2.  The  sample  is 
skewed  toward  innovative  plants.  However,  there  are  a  sizable  number  of 
relatively  standard  units  that  constitute  a  baseline  for  the  analysis. 
About  two-thirds  of  the  plants  are  hydrocarbon  and  hydrocarbon  chemicals 
units.  However,  we  do  have  a  number  of  other  kinds  of  plants,  including 
some  specialty  chemicals,  some  chlorine-based  chemicals,  and  some 
minerals.  For  each  plant  in  the  data  base,  there  is  a  wealth  of 
information  that  describes  various  aspects  of  the  plant  and  of  project 
outcomes.  Data  include: 

•  Technical  characteristics  of  th°  plcnt--in  particular,  items 
that  measure  the  degree  to  which  the  plant  is  innovative, 

•  Cost  estimates,  the  project  stage  at  which  they  were  made,  and 
what  they  include, 
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•  Actual  plant  costs, 

•  Nature  of  any  management  problems, 

•  Operating  data  after  startup. 

Of  particular  importance  to  our  subsequent  analysis  were  the 
details  we  were  able  to  obtain  on  tha  degree  to  which  a  project  had  been 
defined  when  an  estimate  was  made  and  how  inclusive  of  various  cost 
items  the  estimates  were.  Companies  were  also  willing  to  give  us 
information  on  the  status  of  process  development  at  each  estimating 
point . 

ANALYZING  COST  GROWTH  IN  PROCESS  PLANTS 

Let  me  now  describe  the  way  in  which  we  examined  the  causes  and 
correlates  of  cost  estimation  error.  Recall  from  Figure  1  that  the 
problem,  on  average,  is  clearly  underestimation  of  cost--which  is,  of 
course,  the  conventional  wisdom.  Virtually  all  of  the  early  estimates 
in  our  data  base  were  less  than  actual  cost  in  constant  dollar  terms. 
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To  begin  our  analysis,  we  removed  chose  items  from  the  actual  costs 
that  could  not  reasonably  be  foreseen  by  a  cost  estimator  in  the  early 
period.  These  we  call  "external  factors":  changes  in  scope, 
unanticipated  inflation,  unanticipated  changes  in  regulatory  standards, 
unusually  bad  weather,  labor  strikes,  shortages  of  equipment  and 
bottlenecks  in  materials  delivery,  etc.  These  kinds  of  events  can  have 
a  severe  effect  on  schedule  and  therefore  on  plant  cost.  We  removed 
such  external  factors  before  examining  cost  estimation  accuracy  because 
the  external  factors  are  not  the  result  of  project-related  decisions  and 
because  they  will  always  contain  a  sizeable  random  element  and  therefore 
obscure,  rather  than  enhance,  our  ability  to  look  at  the  basic  causes  of 
cost  estimation  difficulties. 

In  fact,  these  factors  make  a  relatively  minor  contribution  to 
misestimation.  As  Figure  3  shows,  all  of  these  external  events  taken 
together  account  for  only  about  a  quarter  of  the  average  growth  in  cost 
from  estimates  to  actual  cost  of  plant.  This  is  an  interesting  finding 
because  it  contradicts  the  conventional  wisdom  of  the  1970s  that 
inflation  and  changes  in  regulatory  standards  were  the  primary  drivers 
of  increased  cost  for  process  plants  in  general,  and  synthetic  fuels 
plant  in  particular.  Our  data  show  that  inaccurate  projections  of 
inflation  seriously  contributed  to  underestimation  of  costs  only  in  an 
18-month  period  in  1973-74;  in  contrast,  in  late  1974  and  1975,  the 
inflationary  rate  tended  to  be  overestimated .  The  number  associated  with 
changes  in  regulatory  standards--ll  percent--is  somewhat  misleading. 

Only  a  very  small  number  of  estimates  were  affected  at  all  by  changes  in 
regulatory  standards,  but  those  few  were  affected  very  severely.  It  is 
the  remaining  3/4  of  this  pie,  which  we  call  cost  growth,  that  was  the 
object  of  our  analysis. 

The  next  figure  (.Figure  4;  pos«s  a  strawman  view  of  well-behaved 
cost  estimates.  Early  estimates  will  be  uncertain  in  the  sense  that 
there  will  be  substantial  variation  around  the  actual  cost  of  plant,  but 
that  uncertainty  will  decline  in  a  reasonably  systematic  way  as  the 
project  moves  toward  completion. 
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Figure  5  shows  l1i.il  cosl  estimates  do  not  really  behave  in  this 
fashion.  The  early  estimates  are  indeed  more  uncertain  in  the  sense  of 
showing  greater  variance  around  the  average,  but  there  is  a  systematic 
tendency  for  the  estimates  to  be  lower  than  actual  costs,  and  the 
variation  around  the  average  underestimation  is  so  large  that  the 
estimates  are,  when  considered  in  just  this  raw  distribution,  worthless 
from  the  point  of  view  of  predicting  actual  plant  cost. 

Our  analytic  goal  was  to  debias  the  cost  estimates.  Our  approach 
was  straightforward.  Based  on  studies  of  weapon  systems  and  our 
discussions  with  industry  personnel  that  preceded  the  data  collection 
phase,  we  had  formed  hypotheses  about  the  factors  that  caused 
underestimation  of  costs.  The  data  from  industry  enabled  us  to  develop 
quantifiable  measures  for  these  factors,  which  we  then  regressed  against 
the  degree  to  which  the  estimates  for  the  plants  in  our  data  base 
deviated  from  actual  costs. 

Figure  6  shows  the  results  of  this  regression  analysis.  This  is 
the  set  of  factors  that,  taken  together,  form  the  best  set  of  predictors 
of  the  degree  of  cost  growth  from  our  initial  sample.  In  summary,  cost 
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FACTORS  ASSOCIATED  WITH  COST  GROWTH 
OF  PROCESS  PLANTS:  RESULTS 
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growth  can  be  predicted  with  substantial  accuracy  using: 

•  two  measures  of  unproven  technology, 

•  the  complexity  of  the  plant, 

•  the  extent  to  which  the  plant  and  project  have  been  defined, 
and 

•  the  degree  of  detail  incorporated  in  the  cost  estimate. 


There  is  an  additional  effect  that  involves  the  interaction  between 
whether  or  not  one  is  in  the  R&D  stage  of  process  development  when  an 
estimate  is  made  and  the  degree  of  definition.  We  developed  a  scale 
that  defines  the  degree  of  project  definition.  As  shown  in  Figure  7,  a 
fully  defined  plant  is  a  2  on  the  scale  and  a  minimally  defined  plant  is 
placed  at  8.  The  scale  is  based  on  the  degree  of  engineering  completed 
when  the  estimate  was  made  and  several  measures  of  site-definition.  As 
you  can  see,  there  are  greater  improvements  in  estimates  to  be  gained 
from  bettor  project  definition  for  projects  in  the  R&D  stages  than  for 
projects  based  on  more  mature  technology.  On  the  oilier  hand,  move 
mature  technology  Is,  of  course,  associated  with  less  underestimation  of 


RELATIONSHIP  OF  PROJECT  DEFINITION  TO  COST 
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cost.  What  is  important  here  is  that  even  those  plants  that  incorporate 
little  or  no  new  technology  are  poorly  estimated  when  the  level  of 
definition  is  low. 

net  me  illustrate  this  last  point  with  the  following  figure  that 
separates  the  average  estimating  error  for  plants  with  less  than  25 
percent  of  their  average  capital  cost  in  new  technology  from  the  error 

for  plants  with  a  greater  amount  (see  Figure  8).  As  you  can  see, 

although  the  more  standard  plants  are  considerably  better  estimated,  the 
estimating  errors  are  still,  on  average,  quite  large. 

I  have  been  discussing  in  a  very  abbreviated  manner  only  one  aspect 

of  our  analysis  in  the  Pioneer  Flants  Study.  Petails  of  the  entire 

analysis  and  of  the  estimating  equation  derived  from  it  can  be  found  in 
Undefst  and  mg  Cost  Growth  and  Perfo:  r.ancc  Shortfalls  in  Pioneer  Process 
Plants.  3 

Recall  that  the  estimating  eriorn  for  the  plants  in  our  data  base 
were  very  large,  especially  for  early  estimates  of  plants  incorporating 

3  L.W.  tier  i  ow ,  K.E.  Phillips,  and  C.W.  Myers,  Understand  ing  Cost 
vtuwLu  titiJ  rti  f ui  /».»// cf?  Shot  i  falls  m  rjunccr  rruccss  rittms t  iht?  Karid 
Corporation ,  R- 2^69 -DOR ,  September  1981. 


new  technology.  Ouir  goal  was  to  develop  a  parametric  cost  model  that 
would  debias  the  estimates  and  provide  a  reasonable  confidence  range. 

The  accuracy  of  the  cost  model  is  displayed  on  the  following  chart  (see 
Figure  9).  Using  the  model,  one  can  recalibrate  over  80  percent  of  the 
estimates  to  within  plus  or  minus  10  percent  of  the  actual  cost  of 
plant.  (Subsequent  to  publishing  those  findings,  we  have  run  blind  tests 
cf  the  model  with  about  a  dozen  additional  facilities.  These  additional 
observations  have  fallen  within  the  standard  error  of  8.3  percent  and 
strengthen  our  confidence  in  the  model's  accuracy.) 

WHY  HAS  THE  COST  ESTIMATION  PROBLEM  PERSISTED? 

When  we  have  briefed  our  results  to  companies  that  supplied  data 
for  this  study,  they  have  repeatedly  raised  the  same  question:  Why 
haven't  we  learned?  Why  do  we  have  the  same  sorts  of  problems  today 
that  we  had  IS  years  ago?  The  evidence  we  have  collected  does  indeed 
suggest  that  despite  new  cost  estimating  techniques,  new  tools  (such  as 
the  computer),  and  presumably  better  data  bases,  cost  estimates  today 
are  no  more  accurate  thf.n  in  the  1960s.  Figure  10  shows  the 
relationship  between  estimation  errors  and  years  for  the  period  1969  to 
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ACCURACY  OF  COST  GROWTH  MODEL 
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1077.  No  trend  toward  improvement  can  be  found.  These  data  suggest 
that  for  every  company  that  has  improved  its  performance,  another 
company's  accuracy  has  declined. 

We  can  only  speculate  about  why  estimates  haven’t  become  more 
accurate,  but  we  can  make  some  reasonable  inferences.  The  causes  fall 
into  two  basic  categories:  problems  that  derive  from  the  estimating 
process  itself  and  problems  that  have  their  source  in  management.  The 
latter  1  will  argue  are  ultimately  more  important  and  more  difficult  to 
c  >  rcumvent . 

Difficulties  in  the  estimating  process 

Estimating  a  major  capital  project  is  an  extremely  complex  task. 
Without  reviewing  the  difficulties  of  cost  est imat ionfc ,  1  believe  that 
the  key  problems  are- 

fc  For  two  discussions  of  estimating  problems,  see  J.W.  Hackney, 

Cnnt rol  and  Management  of  Capital  Projects ,  John  Wiley  and  Sons,  New 
York,  i9r>5;  jud  F  V.  Merrow,  S.W.  Chapel,  and  C.  Worthing,  A  Keview  ot 
Cost  E im.it  ion  in  New  Technologies:  Implications  for  Energy  Process 
Plants,  Th"  Rand  Corporation,  R-2>j81-D0E,  July  1979. 
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•  The  Inherent  biases  of  the  problem.  When  project  costs  are 
estimated  using  conventional  engineering  approaches,  the 
opportunities  to  underestimate  will  always  exceed  the 
opportunities  to  overestimate.  This  will  tend  to  be  more  true 
for  innovative  plants  because  more  problems  remain  unresolved 
at  every  point  than  is  the  case  with  more  standard  units .  On 
average,  early  estimates  based  on  conventional  estimating 
techniques  will  be  biased  low  for  all  plants  simply  because 
more  potential  sources  of  misestimation  remain  unidentified  in 
the  early  stages  and  sources  of  underestimation  always  tend  to 
predominate.  It  is  not  surprising  then,  that  early  estimates 
for  innovative  facilities  are  typically  the  most  underestimated 
of  actual  costs. 

There  are  ways  to  address  the  deficiencies  inherent  in  conventional 
approaches  to  estimation,  including  the  sort  of  approach  taken  in  Rand's 
work.  These  approaches  generally  require  extensive  data  to  establish 
although  they  are  frequently  quite  inexpensive  to  apply  once  developed. 
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•  AtLempLs  to  nconom i/.e .  Because  of  L lie  inherent,  difficulties  in 
preparing  good  engineering  csl  1 mates ,  attempts  to  cut  corners 
and  prepare  estimates  very  inexpensively  or  quickly  will  almost 
always  result  in  underestimation  when  conventional  estimating 
techniques  are  being  employed.  Of  course,  it  is  not  really  the 
estimator's  Lime,  but  the  time  necessary  to  produce  reliable 
inputs  for  the  estimator  that  is  expensive.  Our  work  shows 
very  clearly  that  after  the  effects  of  "pioneering"  are 
controlled  for,  the  biggest  difference  between  the  accuracy  of 
estimates  produced  by  various  companies  is  the  willingness  to 
spend  money  early  on  to  define  the  project.  Unfortunately, 
constant  vigilance  is  necessary  to  keep  the  quality  of  inputs 
consistently  high. 

Managerial  Obstacles  to  Estimation  Accuracy 

Although  the  obstacles  posed  by  the  estimating  process  itself  are 
real  enough,  they  cannot  be  more  than  a  necessary  condition  of  our 
failure  to  advance  significantly  the  estimating  art  over  the  past 
decade.  In  fact,  these  obstacles  arise  in  the  estimating  process  in 
part  because  of  management  decis ions--both  explicit  and  implicit. 

In  the  explicit  category  is  the  key  fact  that  organizational 
incentives  often  work  against  realistic  estimates.  Whenever  an  estimate 
is  prepared,  there  is  someone  in  the  organization  who  wishes  it  were 
lower.  Especially  if  an  estimate  is  more  than  the  project  advocates 
expected,  the  estimator  or  his  boss  will  be  forced  to  justify  why  the 
number  shouldn't  be  less.  (Of  course,  if  the  project  advocates  prepare 
the  estimate  without  some  external  check,  then  management  gets  what  it 
deserves.)  Only  rarely  will  there  be  countervailing  pressure  to  justify 
why  the  estimate  shouldn't  be  higher. 

Less  obvious,  but  more  damaging,  is  the  fact  that  management 
appears  to  be  unwilling  to  invest  in  remembering  the  past.  Among  the  30 
or  so  owr  -  companies  that  I  know  reasonably  well,  1  have  yet  to 
encounter  a  senior  manager  who  did  not.  bemoan  inaccuracies  in  cost 
estimates  And  yet  not  more  than  a  handful  of  those  companies  are 
willing  to  invest  seriously  in  internal  efforts  to  fashion  better 


estimating  tools.  In  the  course  of  collecting  data  for  our  analyses, 
have  discovered  that  many  companies  could  not  produce  relevant  costing 
data  for  even  one  plant.  Among  those  who  did  keep  full  records,  the 
data  were  often  archived  in  a  way  that  rendered  them  inaccessible.  I 
have  always  found  it  ironic  that  companies  willing  to  commit  many 
millions  annually  to  process  R&D  in  the  hopes  of  lowering  production 
costs  seem  unwilling  to  commit  even  nominal  amounts  to  research  on 
better  techniques  for  estimating  the  projects  they  build. 


